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Abstract.  By combining  the  principles  of  known  factoring 'algorithms  we  obtain 
some  improved  algorithms  which  by  heuristic  arguments  all  have  a  time  bound 
Q(exp  vfln  n  In  In  nj  for  various  constants  c  >  3.  In  particular,  Miller's  method 
of  solving  index  equations  and  Shanks  method  of  computing  ambiguous  quadratic 
forms  with  determinant  —n  can  be  modified  in  this  way.  We  show  how  to  speed 
up  the  factorisation  of  n  by  using  preprocessed  lists  of  those  numbers  in  [— u,  u] 
and  (#  —  u,  n  +  u],  0  s,  <  u  <  <  n  which  only  have  small  prime  factors.  These 
lists  can  be  uniformly  used  for  the  factorisation  of  all  numbers  in  [n  —  u,  n  + 
u).  Given  these  lists,  factorisation  takes  0(exp[2(ln  njV*(ln  In  nj?/*])  steps.  We 
slightly  improve  Dixon's  rigorous  analysis  of  his  Monte  Carlo  factoring  algorithm. 
We  prove  that  this  algorithm  with  probability  1/2  detects  a  proper  factor  of  every 
composite  n  within  0(exp  Vfelnnlnlnn'i)  steps.  \ 


This  work  was  done  in  summer  1980  during  a  stay  at  the  Stanford  Computer 
Science  Department.  Preparation  of  this  report  was  supported  in  part  by 
National  Science  Foundation  grant  MCS-77-23738. 


1.  Introduction  and  Summary. 

Recently  the*interest  in  factoring  integers  dramatically  increased  since  the 
security  of  the  RSA  public  key  cryptosystem  mainly  relies  on  the  difficulty  of 
factoring  large  integers,  see  Rivest  et  a!.  (1978).  The  problem  of  factoring  integers 
is  one  of  the  classical  computational  problems  in  mathematics.  Gauss  quoted  it  as 
one  of  the  most  important  and  most  useful  problems  of  arithmetics.  Only  modest 
progress  has  been  made  from  the  factoring  methods  known  to  Gauss  and  Legendre 
to  the  most  efficient  algorithms  known  today.  In  fact  almost  no  new  ideas  came 
up,  the  progress  mainly  relies  on  more  efficient  programming  and  the  use  of  faster 
computing  machinery.  Landmarks  of  this  progress  have  been  the  factoring  of  the 
Fermat  numbers  Ft  =  22  +  1  by  Morrison  and  Brillhart  (1975)  and  recently  the 
factoring  of  Fs  =  22  -f-  1  by  Brent.  The  theoretical  progress  mainly  concerns  a 
better  understanding  and  a  more  detailed  analysis  of  the  known  methods.  Also, 
with  the  evolution  of  the  theory  of  computational  complexity  there  evolved  an 
increasing  interest  in  asymptotical  runtimes  of  algorithms.  We  will  continue  in 
this  direction,  too. 

In  order  to  factor  n,  or  equivalently  to  solve  x2  —  —  amodn,  Gauss  (Artikel 
327)  makes  extensive  use  of  the  theory  of  quadratic  forms.  The  usefulness  of 
quadratic  residues  mod  n  which  are  small  or  only  have  small  prime  factors  has 
been  known  long  ago.  Gauss  (Artikel  328)  gives  a  method  to  construct  such 
residues  to  with  w  =  0(\/n)  by  means  of  quadratic  forms.  Legendre  already  used 
the  continuous  fraction  expansion  of  \/n.  The  more  recent  factoring  algorithms 
of  Morrison,  Brillhart  (1975),  Shanks  (1971,  1974),  J.  P.  C.  Miller  (1975)  are 
all  refinements  and  variations  of  these  old  ideas.  This  will  become  clear  by 
a  comparative  study  of  these  algorithms,  including  proper  modifications  and 
improvements. 

From  the  theoretical  point  of  view  Dixon  (1978)  achieved  a  major  step.  He 
proposed  a  probabilistic  factoring  algorithm  and  gave  a  rigorous  proof  that  this 
algorithm  for  every  composite  number  n  with  probability  1/2  detects  a  proper 
factor  of  n  within  0(exp(4v^ln  n  In  Inn))  steps.  Section  2  contains  an  outline 
of  Dixon’s  analysis  together  with  some  improvements.  In  fact  we  decrease  the 
constant  4  to  \/6.  If  in  addition  quadratic  residues  mod  n  are  constructed 
via  Legendre's  continuous  fraction  method  then,  under  reasonable  assumptions, 
we  obtain  the  time  bound  0(exp\/31nnlnlnn)  for  a  tuned  up  version  of  the 


Morrison-Brillhart  algorithm. 

In  Section  3  we  analyze  J.  P.  C.  Miller's  method  of  using  the  solutions  of 
index  equations.  We  point  out  that  this  is  not  an  independent  method  but  rather 
a  modification  of  solving  z2  =  y 2  mod  n  by  combining  congruences  modn.  Under 
reasonable  assumptions  we  obtain  a  time  bound  0(exp  v^4.5  In n  lnln  n).  However 
this  algorithm  might  be  the  most  efficient  one,  if  one  likes  to  factor  many  numbers 
in  a  small  region.  The  reason  is  that  this  algorithm  uses  lists  of  those  numbers  in 
[ — u,  u]  and  (n  —  u,  n  +  u]  which  only  have  small  prime  factors.  These  lists  can 
be  uniformly  used  for  the  factorization  of  all  numbers  in  [n  —  u,  n  -f-  u]. 

In  Section  4  we  modify  Shanks  (1971)  method  of  factoring  n  via  the  con¬ 
struction  of  ambiguous  quadratic  form*  with  determinant  — n.  Our  modification 
relates  this  algorithm  to  the  previous  ones  and  in  particular  to  the  Morrison- 
Brillhart  algorithm.  Under  reasonable  assumptions  we  obtain  the  time  bound 
0(exp  \/31nnlnlnn). 

This  latter  algorithm,  the  Morrison-Brillhart  algorithm  and  the  Schrocppel 
algorithm  (see  Monier  (1980))  are  the  asymptotically  fastest  known  factoring 
algorithms.  A  rough  analysis  slightly  favors  Schroeppel’s  algorithm  since  under 
reasonable  assumptions  we  obtain  a  time  bound  0(exp(1.5\/ln  nlnln  n)).  There 
is  however  an  additional  speed  up  for  the  other  two  algorithms,  due  to  the  fact 
that  about  half  of  the  primes  cannot  occur  as  factors  of  the  residues  occuring  in 
the  algorithm.  This  effect  is  difficult  to  analyze  but  might  well  dominate  for  all 
reasonably  sized  n  over  the  small  difference  \/3—  1.5  in  the  exponent. 

We  should  at  least  mention  the  important  algorithms  of  Pollard  (1975)  and  of 
Schrocppel  (see  Monier,  1980)  which  are  not  included  in  this  comparative  study. 
For  more  complete  surveys  on  factoring  algorithms  we  recommend  Guy  (1975), 
The  Art  of  Computer  Programming,  Vol.  2  by  D.  Knuth  (in  particular  the  1980 
edition),  and  the  thesis  of  L.  Monier  (1980). 
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2.  A  Rcflncd  Analysis  of  Dixon's  Probabilistic  Factoring  Algorithm. 

So  far  the  asymptotically  fastest  run  time  of  a  factoring  algorithm  has  been 
proved  by  Dixon  (1978).  Given  a  composite  number  n,  this  algorithm  finds  a 
proper  factor  of  n  with  probability  1/2  within  0(exp(4\/ln  n  In  In  n ))  steps.  In 
denotes  the  “logarithmus  naturalis"  with  the  Eulerian  number  t  as  base  and  exp 
is  the  inverse  function  to  In.  Dixon  mainly  applies  the  method  of  "combining 
congruences”  to  generate  solutions  of  z2  =  y2modn.  In  Sections  3  and  4  we  will 
see  that  this  technique  can  well  be  combined  with  factoring  algorithms  proposed 
by  J.  P.  C.  Miller  (1975)  and  D.  Shanks  (1971).  We  give  an  outline  of  Dixon’s 
algorithm  with  an  improved  analysis.  We  decrease  the  constant  4  in  Dixon’s  bound 
to  V&.  The  improved  theoretical  time  bound  results  from  a  tighter  lower  bound 
on  the  number  of  quadratic  residues  modn  which  can  be  completely  factored  over 
small  primes  (Lemma  1)  and  a  specific  method  for  detecting  small  prime  factors. 
Here  we  do  not  focus  on  designing  the  most  practical  algorithm  but  we  like  to 
prove  a  rigorous  asymptotical  time  bound  as  small  as  possible. 

Dixon’s  Algorithm, 
begin  input  n 
stage  1  v  =  [n1/2rJ 

comment  the  optimal  choice  of  r  €  «V  will  be  made  below. 

Form  the  list  P  of  all  primes  <  v  .  P  =  {/>?., . ..,pff(v)}. 
if  3 pi  £  P  :  Pi\n  then  print  p,-  stop 
B  :=0 

stage  2  Choose  z  £  [1,  n]  at  random  and  independently  from  previous  choices  of 
z. 

t o  :=  z2  mod  n  with  0  <  w  <  n 

stage  3  Compute  a  =  (a,-  £  M  |  1  <  i  <  7r(u))  and  w *  with  w  =  to*  n,-<w(v)Pi< 
and  Vp  6  P'  p  does  not  divide  w* . 

test  1  if  [w*  1  or  g  =  Qmod  2]  then  goto  stage  2 

B:=SU  {a},  :=  z 

Try  to  find  a  nontrivial  solution  of 

£/aq  =  0mod2  46(0,1).  (1) 

t6B 
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test  2  if  there  is  no  nontrivial  solution  then  goto  stage  2 

TT  TT  2 

x  ■=  n /.=i  y  :=  n«,w  * 

comment  [The  construction  implies  z2  —  y2modn;  in  case  z  ^  ±y 
mod  n,  gcd(x  ±  y,  n)  are  proper  factors  of  n.J 


test  3  if  z  7^  ±y  mod  n  then  print  gcd(x  ^  y,  n)  stop 
Choose  the  first  a  G  B  such  that  /A  =  1 . 

B  :=  B  —  {a},  goto  stage  2 

end 


Obviously  a  proper  factor  of  n  has  been  found  as  soon  as  test  3  succeeds.  In 
the  following  analysis  of  the  algorithm  we  suppose  that  n  is  an  odd  number  with 
prime  factor  decomposition. 


d 

n—  JJ  q*1  li  >  1  and  d>  2. 

t=i 

Clearly  the  cases  that  n  is  even  or  a  pure  prime  power  can  easily  be  handled 
in  advance.  The  following  facts  are  due  to  Dixon. 


Fact  1.  prob(z  =  ±ymodn  within  test  3)  =  21~d  and  the  corresponding  events 
for  distinct  passes  of  test  3  are  mutually  independent. 

Proof.  Consider  the  last  chosen  z  and  w  =  z2modn.  We  prove  that  there 
are  exactly  2d  distinct  z,-,  :  =  l,...,2d,  such  that  z2  =  tamodn.  Clearly  Z\, 
the  mu’tiplicative  group  mod  n,  is  a  direct  sum 


*;-©**• 

1 

For  each  i  there  are  exactly  two  distinct  solutions  t,-  =  u,,  t>,  of  t2  =  tnmodq*‘. 
Then  by  the  Chinese  remainder  theorem  the  z,  correspond  in  one-one  manner  to 
the  2d  elements  in  {u3 ,  ui }  X  •  •  •  X  {ud,  Vd } .  Now  each  of  z\ , . . . ,  z2*  is  equally 
likely  to  be  chosen  for  z.  The  values  of  /4  and  y  do  not  depend  on  the  choice 
of  z  G  {zi,...,Z2*}»  only  z  =  11/,=  1*4  depends  on  this  choice.  Observe  that 
the  value  /4  corresponding  to  z  =  z&  must  be  1,  otherwise  the  algorithm  would 


5 


pass  test  3  without  choosing  this  final  z.  Therefore  the  2*  choices  for  z  yield 
2d  distinct  values  for  z  and  exactly  two  of  them  imply  z  =  ±ymodn.  This 
evaluates  the  probability  that  “z  =  ±ymodn  during  test  3"  to  2l~d.  Since 
our  analysis  is  completely  based  on  the  last  chosen  z,  it  is  clear  that  the  distinct 
events  of  “test  3  succeeding"  are  mutually  independent.  B 

Let  T(n)  be  the  total  time  of  the  algorithm  and  let  T^ri)  be  the  time  till 
the  first  pass  of  test  3.  We  count  arithmetical  steps  mod  n  as  single  steps.  T(n), 
Ti(n)  are  random  values  depending  on  the  random  variables  z  of  stage  2.  Fact  1 
immediately  implies: 

Fact  2.  £(7(n)]  =  (1  -  21~<i)-1£(73(n)]  <  2E[T3(n)]. 

Here  2s(X]  denotes  the  expectation  of  the  random  value  X.  Let  Ti(n)  (T2(n), 
resp.)  be  the  time  spent  from  any  entering  of  stage  2  till  the  first  pass  of  test  1 
(test  2,  resp.)  without  counting  the  steps  used  to  solve  the  various  linear  systems 
of  equations  (1).  Since  a  linear  dependence  of  the  a  with  a  E  B  must  exist  as 
soon  as  #2?  >  7r(v)  +  1  =  0(u/lnv)  it  follows  that  there  are  almost  ?r(u)  +  1 
passes  of  test  2  between  two  consecutive  passes  of  test  3.  Hence 

Fact  3.  E[T3(n)]  <  (ar(w)  +  l)£[72(n)]  -f  0(rr(t;)3). 

Here  0(tt(v)3)  bounds  the  steps  to  solve  all  the  linear  systems  (1)  occuring 
in  the  various  passes  of  stage  3.  Indeed  this  task  amounts  to  solve  one  system  of 
linear  equations  with  7r(v)  -f  1  unknowns.  In  order  to  analyze  2?[72(n)]  we  define 

Q  :=  {set  of  quadratic  residues  mod  n)  f|  Z\ 

T{n,  v)  :=  {r  6  [1,  n):  all  prime  factors  of  r  are  <  u} 

M{n,v)  :=  {z  6  [l,n]:  2r2modn  E  QflT(n,  t»)> . 

Fact  4.  2?(7a(n)]  <  0(F(7t(n)]n/#A/(n,  w)). 

Proof.  We  clearly  have  prob[ w*  =  1)  >  #M(n,  v)/n  and  it  can  easily  be 
seen  that  prob(a  =  0  mod  2)  is  negligibly  small.  Hence  test  1  will  almost  be  passed 
about  n/(#M(n,v))  times  between  two  passes  of  test  2. 

Ti(n)  depends  on  how  the  factorization  of  w  over  the  prime  base  P  is  done. 
A  crude  way  is  as  follows: 
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w*  :=  w 

for  all  p  6  P  do 

(while  p  |  w*  do  w*  :=  t o*/p] 

This  yields 

Fact  5.  E\Ti(n)\  <  n(v)  +  logn. 

Here  logn  bounds  the  number  of  multiple  prime  factors  of  n  according  to 
their  multiplicity. 

So  far  Facts  1-5  yield  under  the  assumption  logn  <  7r(u): 

+  (2) 

and  it  remains  to  prove  a  sharp  lower  bound  on  #M(n,  v).  This  will  be  our 
main  improvement  over  Dixon’s  analysis.  Let  k:  Z*  -*  {±1}*  «  ®(LjZ2  be 
the  quadratic  character,  defined  as  follows.  For  c,*)  6  let 

/c(oi . ad)  —  (ei,...,Cd)  with  e,-  =  (^).  By  definition  the  Jacobi  symbol  (*) 

is  1,  ( — 1,  resp.)  if  6  is  a  quadratic  residue  (non-residue)  modq.  It  is  well  known 
that  k  :  Z*  -*  is  a  group  homomorphism  and  a  E  Q  iff  *(a)  is  the  group 

unit  (1,1,...,  1)  €  {±l}d. 

Lemma  1.  #M(n,  v)  >  ?r(v)2f/(2r)!  for  all  natural  numbers  r  with  v2r  <  n 
provided  all  prime  factors  of  n  are  >  v. 


Proof.  Let  Tr{m,v)  :=  {w  6  (l,m)  |  w  =  npt<vPi‘  A  ^  r)-  Since 

all  prime  factors  of  n  are  >  v  we  have  Tr(\/n,  v)  C  Z*.  We  partition  T,(y/n,  v) 
into  classes  T,-,  i  =  l,...,2d  according  to  the  2d  possible  values  of  k.  Then 

d 

U  TiTiCT2r{nlv)nQ. 

t=i 


Therefore 


#M(n,  v)  >  2i#(T2r(n,v)nQ) 


(3) 


7 


i 


Here  (#7\)2  counts  the  number  of  ordered  pairs  (wt,  w2)  6  7V  X  7V  and  (2r)»  /(r!)2 
bounds  for  each  w  6  Q  the  number  of  distinct  pairs  (u>i  ,  ty2)  6  (J|7V  X  TV  that 
yield  the  product  u>ity2  =  w.  The  Cauchy  Schwarz  inequality  implies 

2<#r.)2  2~d(T,  #T<)  =  2-d*Usfc,  vf 

»'=1  '  »  ' 

(use  ^  *  51 -  (C  UjU,-)2  with  u,-  =  #7V,  y,-  =  1). 

t  «  t 

Obviously  we  have  #7V (\/^,  t;)  =  (’r(w)+r)  >  r(y)r/r!,  since  (,r(v)+r)  is  the 
number  of  possibilities  of  choosing  with  repetitions  r  elements  out  of  tt(v).  Finally 
we  obtain  from  (3),  (4): 

.....  .  .  ^  r!2  ;r(y)2r  r!2  7r(v)2f 

#W(B-  ^  -  #r'(,/5’ V)  (20!  -  TP"  (20!  =  W  8 


Putting  (2)  and  Lemma  1  together  we  obtain 


£|T(n)]  =  +  »(»))) 


provided  logn  <  jr(y)  and  v2r  <  n.  Using  v  =  n1/2r,  the  prime  number  theorem 
in  the  form  y/lnv  <  7r(y)  <  2v/\nv  and  Stirling’s  formula 


(2r)!  =  0(\/2r(2r)2re— 2r) 


we  obtain 


We  choose  r  6  M  as  to  minimize  n^^lnn)2’.  This  implies 


This  finally  yields 


2?(T(n)]  =  0 


y/2re 


2r 


Inin  n 


exp  Vs  Inn  In  Inn 


=  0(exp\/81nnlnlnn). 


(5) 


The  asymptotic  behavior  of  this  bound  is  quite  attractive  for  excessively  large 
n:  n  can  be  factored  within  steps  with  c(n)  -+  0  for  n  — ►  oo.  However, 
for  reasonably  sized  values  the  exponent  c(n)  is  not  much  smaller  than  0.5  and 
the  algorithm  therefore  hardly  beats  straightforward  factoring  algorithms.  For 
instance  in  the  range  n  «  c200  we  choose  r  =  4  and  (5)  yields  £[T(n)]  <  e84  = 
n0-42. 

Can  the  above  analysis  of  Dixon’s  algorithm  still  be  refined  leading  to  a 
constant  in  the  exponent  which  is  smaller  than  \/8  ?  We  discuss  two  main  points, 
(a)  the  tightness  of  our  lower  bound  on  #M(n,  v)  in  Lemma  1,  (b)  the  use 

of  more  sophisticated  factoring  algorithms  for  factoring  w  over  the  prime  base  P 
in  stage  2. 

We  clearly  have  #M(n,  v)  <  tl>(n,v) #{w  6  [I,  nj:  all  prime  factors  of  tv 
are  <  v }.  The  asymptotic  behavior  of  t^{n,v)  has  been  analyzed  for  a  long  time. 
De  Bruijn  (1966)  proved 


yinyj 

[1  +  0(lny)_1  -fO(lnz)-1  +0(1  +  n)-1] 


with  u  =  Inz/ln  y.  If  this  upper  bound  on  ti>{n,v)  is  used  instead  of  the  lower 
bound  7r(u)2,/(2r)l  with  v2r  <  n,  it  leads  to  the  same  constant  \/8  in  our  time 
bound.  This  shows  that  asymptotically  we  do  not  lose  too  much  by  the  slackness 
of  Lemma  1.  However  for  reasonably  sized  n  the  algorithm  will  perform  somewhat 
better  than  our  rigorous  time  bound  indicates. 

Instead  of  using  within  stage  2  the  straightforward  factoring  algorithm  that 
leads  to  Fact  5  we  could  use  one  of  Pollard’s  algorithms  that  finds  factors  <  v 
of  n  in  about  0(\/u)  steps.  By  computational  experience,  Pollard’s  p-method 
(1975)  detects  factors  <  v  of  n  in  0(\/t;lnv)  arithmetical  steps  mod  n,  see  Guy 
(1975)  and  Knuth  (198C).  This  method  is  highly  practical  although  no  rigorous 


theoretical  time  bound  is  known  so  far.  Recently  Brent  succeeded  in  factoring 
Fs  —  22*  +  1  by  a  variant  of  this  method.  Pollard  (1974)  also  proposed  a  second 
method  with  a  rigorous  time  bound.  He  computes  for  sufficiently  many  small 
a  6  2*n,  gcd(n^^(a^-a-^n)  for  p  =  l,2,...,\/v.  For  fixed  o  these 
gcd- values  can  be  computed  by  the  fast  Fourier  transform  within  0(y/v(\n  v )2) 
steps.  In  total,  Pollard  obtains  a  worst  case  time  bound  0(y°-5+‘)  for  arbitrarily 
small  e  >  0,  but  the  constant  factor,  expressed  by  0,  increases  in  an  unknown 
way  as  e  decreases.  We  give  a  similar  but  slightly  stronger  result. 

Lemma  2.  For  a  ay  fixed  v  the  smallest  factor  <  v  of  n  can  be  found  in 
0(y/v(lnv)2)  arithmetical  steps  modn. 

Proof.  Without  loss  of  generality  we  assume  that  y/v  is  an  integer.  Evaluate 
/(z)  —  II^ (*-*) modn  at  z  =  ty/v  for  t  =  1,2, ...,y/v.  Using  the  fast 
Fourier-transform  this  can  be  done  within  0(\/u(lii  v)2)  arithmetical  steps  mod 
n,  see  e.g.  Borodin,  Munro  (1974),  Cor.  4.5.4.  Then  compute 

t  :=  min{f  <  y/v:  gcd (f[ty/v),n)  >  1} 
i  :=  max{i  <  y/v:  ( ty/v  — :')  j  n) 

Then  ty/v  —  i  is  the  smallest  factor  <  v  of  n.  The  correctness  of  this  procedure 
is  obvious.  Q 

Using  the  above  procedure  in  searching  for  a  prime  factor  <  v  of  to  in  stage 
2,  we  improve  Fact  5  to 

FactG.  Ti(n)  =  0(\/o(In  v)2). 


Now  from  Facts  1-4,  5,  Lemma  1,  v/lnv  <  ?r(y)  <  2vJ\nv  and  Stirling’s 
formula,  we  obtain  for  v  =  n1/2': 

?r(v)\/y(ln  u)2n(2r)!  .  3^ 

£lr(»)]  =  0(  ,V  +  *(v)3 J 

=  0^n3//,<r  lnu  \/2r  e~2r(lnn)2r  +  n3/,2r^j^^  (6) 

We  choose  r  £  M  as  to  minimize  n3/4,(lnn)2r  and  obtain: 


2r  = 


(  3 Inn 
2  In  Inn 


-j-  c  with  |e|  <  1 
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This  finally  yields 


Thus  we  succeeded  in  decreasing  the  constant  in  the  exponential  term  at  the 
expense  of  increasing  the  low  order  factor.  In  the  range  n  «  e200  we  have  r  =  6 
and  (7)  yields  2?[T(n)j  «  e80/15  «  n0,,</15  which  is  only  marginally  better  than 
the  conclusion  from  (5). 

Theorem  1.  For  each  composite  n  let  J?[T(n)]  be  the  expected  time  that  the 
above  algorithm  Hods  a  proper  factor  of  n.  Then  for  all  n 

(1)  EfT^n)]  =  0(exp  VG  In  n  In  In  n ). 

(2)  The  event  that  the  algorithm  does  not  find  a  proper  factor  of  n  within 
A£'[T(n)]  steps  has  probability  <  2~k. 

Statement  (2)  is  an  immediate  consequence  of  the  fact  that  the  distinct  events 
of  “test  3”  (test  1,  resp.)  failing"  are  mutually  independent. 

A  more  practical  factoring  algorithm  is  obtained  if  the  quadratic  residues  w 
in  stage  2  are  produced  via  the  continuous  fraction  method  (see  Morrison  and 
Brillhart,  1975)  which  implies  w  =  0(\/n)  and  if  Pollard’s  />-method  is  used  for 
detecting  small  prime  factors  of  w. 

Under  the  assumption 

(AO)  the  continuous  fraction  of  >/n  generates  quadratic  residues  mod  n  which 
are  uniformly  distributed  in  (l,0(\/n)] 

the  time  bound  (6)  transforms  into  a  time  bound 

£[r(n)]  =  ofn3/4'  In  n  e-f(!n  n)f  +  "3/2r( )  (8) 
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with  r  even,  for  the  Morrison-Brillhart  method.  By  choosing 


r 


2 


3)nn 
In  Inn 


we  obtain 

ft3^r(lnn)'  =  0((ln  n)2  exp  >/3  In  n  In  In  n ) 
n3/2r  _  o(exp \/31nnlnlnn). 

By  (8)  this  implies 

Corollary  1.  [Assume  (AO).}  The  Morrison-Brillhart  method  runs  in  average  time 
0(exp  V3  Inn  In  Inn ) . 

In  particular  (8)  with  r  =  6  implies  is’fTfn)]  «  e56  «  n028  for  n  «  e200. 
However,  by  experience  a  well  tuned  version  of  the  Morrison-Brillhart  method 
behaves  somewhat  better  for  reasonably  sized  n.  Wunderlich  (1979)  obtained 
average  run  times  322  •  n 0152  «  n°  21  for  n  «  1040.  In  fact  there  are  several 
points  where  our  worst  case  analysis  is  too  pessimistic.  The  lower  bound  on 
#A^(n,v)  in  Lemma  1  is  somewhat  too  small.  Moreover,  it  is  known  that  the 
quadratic  residues  generated  by  the  continuous  fraction  method  can  only  have 
prime  factors  p  with  (£)  =  1.  Since  only  about  half  of  the  primes  appear  as 
factors  of  the  w’s,  this  has  the  effect  of  doubling  the  size  7r(u)  of  the  prime  base. 
We  estimate  that  this  increases  the  ratio  of  w's  which  are  completely  factorizable 

over  the  prime  base  by  22f  and  therefore  causes  a  speed  up  factor  of  about  212  « 
n°  041  fQr  n  ^  e200 

Assuming  (AO)  is  only  a  first  imperfect  step  towards  an  analysis  of  the 
Morrison-Brillhart  method.  Indeed  the  continuous  fraction  of  \/n  behaves  too 
uncivilized.  It  should  be  important  for  a  more  rigorous  analysis  to  have  a  lower 
bound  #{p  <  v:  p  prime,  (£)  =  1}  >  with  c  >  0  fixed.  This  would 
ensure  a  sufficiently  large  base  of  small  primes  for  this  method.  It  is  also  unclear 
whether  this  method  finds  each  factor  of  n  equally  likely  or  whether  some  factors 
are  harder  to  find  than  others.  A  similar  situation  will  occur  in  the  discussion  of 
an  analogous  algorithm  in  Chapter  4. 


3.  An  Analysis  and  Revision  of  J.  P.  C.  Miller's  Factoring  Method. 

J.  C.  P.  Miller  (1975)  proposed  a  factoring  method  based  on  the  computation 
of  indices.  We  shall  develop  a  slightly  improved  version  of  Miller’s  method  which 
turns  out  to  be  quite  similar  to  the  previously  analyzed  Dixon  algorithm.  Under 
reasonable  heuristic  assumptions  the  runtime  of  our  version  of  Miller’s  algorithm 
will  be  0(exp  V4  5  In  nlnln  n).  In  particular  Miller’s  method  does  not  yield 
an  independent  factoring  algorithm  but  merely  a  specific  modification  of  the 
method  of  “combining  congruences  modn”.  However,  as  we  shall  point  out,  this 
modification  has  some  decisive  advantages  in  the  case  that  one  likes  to  factor 
many  numbers  in  the  same  range.  So  far  all  known  factoring  algorithms  collect 
data  which  are  only  useful  for  factoring  one  specific  number.  For  instance  the 
congruences  collected  in  Dixon’s  algorithm  cannot  be  used  for  different  n’s.  This 
observation  also  applies  to  the  factoring  algorithms  of  Morrison-Brillhart  (1975), 
Schroeppel  (unpublished,  see  Monier  1980),  Shanks  (1971,  1974),  and  Pollard 
(1974,  1975).  In  our  version  of  Miller’s  method  we  will  collect  products  of  small 
prime  numbers  which  are  near  to  the  number  n  to  be  factored.  These  products 
of  small  primes  can  be  uniformly  used  for  factoring  all  numbers  near  to  n. 

For  a  G  ord(a,  n)  :=  min{i/  J  av  =  Imodn}  is  the  order  of  a  modn. 
X(n)  :=  max{ord(a,  n)  |  a  G  2*}  is  the  order  of  Z‘n.  Let  hi,  hi,...,  hi  be 
a  system  of  independent  generators  of  Z\,  then  for  every  a  G  Z‘n  there  ;s  a 
representation 

t 

o  =  JJ  h™‘  modn 

»'=i 

whore  m{modord(h,-,n)  is  uniquely  determined.  Then  ind(o)  :=  {mi,...,  mi)  is 
called  a  (multi-)  index  of  a. 

Miller  first  tries  to  determine  ord(a,  n)  for  some  small  primes  o  as  follows. 
Every  solution  x  of 

x  •  ind(a)  =  0(mod\(n))  (1) 

is  a  multiple  of  ord(a, n).  Linear  index  equations  mod  X(n)  are  obtained  from 
representations  of  n  as  a  sum  or  a  difference  of  products  of  small  primes.  These 
equations  are  solved  by  Gaussian  elimination  in  order  to  obtain  a  solution  x  of 
(1).  We  have  to  factor  x  in  order  to  determine  ord(a,  n).  Let  ord(a,  n)  =  aV 


13 


with  cij  prime,  then  eventually  gcd(aord^a,n^a/  —  l,n)  will  be  a  proper  factor 
of  n. 

As  an  example,  let  n  =  1037. 

stage  1:  Search  for  many  distinct  representations  of  n  or  multiples  of  n  as  a  sum 
or  difference  of  two  products  of  small  primes.  For  instance  we  have 


*  1037  =  2S5  —  35  i.e. 

=  24  •  5  •  13  —  3 

*  =  2  •  3  •  52  •  7  —  13 
=  210  +  13 

*  =  2235  +  5  •  13 

*  =  3  •  73  +  23 


2S5=  35  modn 
24  •  5  •  13  =  3 modn 
2  •  3  •  52  •  7  =  13 modn 
210=  — 13  mod  n 
2235=  —5  •  13  mod  n 
3  •  73=  — 23  modn 


It  follows  that  there  exist  multi-indices  z,  a,  b,  c,  d,  t  for  —1,  2,  3,  5, 
7,  13  such  that 

8a  -f  c  =  56modX(n) 

4a  +  c  +  t  =  b  mod  X(n) 
a-fi-f2c-f-d  =  e  mod  X(n) 

10a  =  z  +  c  mod  X(n) 

2a  -f-  56  =  2  +  c  +  t  mod  X(n) 
b  -f  3d  =  z  +  3a  mod  X(n) 

stage  2:  Gaussian  elimination  yields 

120a  =  OmodX(n). 

Hence 

2120  =  1  modn, 

which  means  ord(2,  n)  |  120. 

The  prime  factors  of  120  are  2,  3,  5  and  since  260,240,224  ^  1  modn 
we  know  ord(2,  n)  =  120. 

stage  3i  proper  factors  of  n  are  found  as 

gcd(260  —  l,n)  =  61 
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gcd(240  —  1,  n)  =  61 
gcd(224  —  l,n)  =  17. 


The  main  critical  points  of  this  algorithm  are  the  following: 

stage  1  How  can  we  generate  sufficiently  many  congruences  such  that  elimination 
works  in  stage  2? 

stage  2  Suppose  a  multiple  x  of  ord(a,n)  has  been  found,  what  is  the  chance  to 
find  sufficiently  many  prime  factors  of  x? 

stage  3  will  fail  to  find  a  proper  factor  of  n  =  JJjL !  p\l  if  ord(a,  pf‘),  i  =  1, . . . ,  d 
all  coincide. 

The  following  modification  circumvents  the  traps  of  stages  2  and  3. 

In  our  example  for  n  =  1037  we  obtain  by  multiplying  the  marked  con¬ 
gruences: 

2U375374  =  233s5  •  132  modn. 


Since  no  prime  of  our  base  divides  n,  this  yields 

28325274  =  lc2  modn. 


From  24  •  3  •  5  •  72  =  353  mod  n  we  obtain 

3532  =  132  modn 
which  gives  us  the  proper  factors 

gcd(353  —  13,  n)  =  17 
gcd(353+  13,  n)  =  61. 

A  formal  description  of  our  method  is  as  follows. 

begin  input  n 

v  :=  n1/2',  u  :=  n^2f 

comment  the  optimal  choice  of  r  and  d  will  be  made  below 
Form  the  list  P  =  {po.Pii •••»?»(*)}  of  primes  <  v,  including  po  — 
—1 
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if  3p,-  €  P,  i  >  1 :  p,- 1  n  then  print  p,-  stop 
stage  1  Compute  the  lists 


L  := 


M  <  «.  ®  =  HiP?  ) 
A  =  (o,-  j  0  <  t  <  7r(u))  J 


B 


l  ;=  |(n4-tu,6) 
:=  {(O.B 


m  <«.*»= n,p!‘  \ 

4=  (6.  I  0  <  t  <  x(«/))J 

7^  Omod2 

3to  :  (to,  o)  G  L  A  (n  +  ty,  6)  6  L 


) 


stage  2  Find  a  nontrivial  solution  |  (s,&)  6  B)  of 

X)  /(s,a(5i)  =  Omo d2,  /(^e{0,l>.  (2) 


test  2  if  no  solution  exists  then  increase  u  goto  stage  1 

x  JT 

»<*(v) 

comment  the  construction  implies  a;2  =  y2  mod  n. 

test  3  if  x  iymodn  then  print  gcd(z  ±  y,n)  stop 
Choose  the  first  (g,£)  £  B  such  that  /(^y  =  1 
B  :=  B  —  {(a,£)}  goto  stage  2. 
end 

This  algorithm  is  virtually  very  similar  to  the  one  of  Dixon,  and  on  the  other 
hand  it  is  an  improved  version  of  Miller’s  method.  Clearly  the  linear  system  (2) 
has  a  nontrivial  solution  as  soon  as  #B  >  2(tt(v)  -f 1).  Compare  this  with  the  use 
of  the  congruences  in  Miller’s  method:  if  the  vectors  in  B  are  linearly  independent, 
then  Gaussian  elimination  in  Miller’s  method  works  as  soon  as  #B  >  ff(t/)  +  1. 
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However,  linearly  dependent  vectors  in  B  are  useless  in  Miller’s  method  and  must 
be  discarded.  It  is  not  easy  to  analyze  the  ratio  of  linear  dependencies  occuring  in 
B.  These  linear  dependencies  will  speed  up  our  algorithm  while  they  slow  down 
Miller’s  method. 

Even  if  Gaussian  elimination  succeeds  in  Miller’s  method  there  are  still  further 
traps  in  stages  2  and  3  of  this  method,  in  particular  the  required  factorization  of  z 
and  ord(o,  n)  is  a  serious  obstacle.  On  the  other  hand  the  only  remaining  trap  in 
our  algorithm  after  solving  the  linear  system  is  the  test  “z  ^  ±ymodn?”  Here 
the  argument  of  Fact  1,  Section  2  indicates  that  this  test  fails  at  a  frequency  21— d 
when  n  has  d  distinct  prime  factors.  However  we  are  no  more  able  to  provide  a 
rigorous  proof. 

The  time  analysis  of  our  algorithm  will  be  based  on  the  following  assumptions. 

(Al)  The  ratio  of  the  number  of  times  of  “test  3  failing"  to  “test  3  succeeding” 
is  bounded. 

(A2)  The  numbers  which  are  completely  factorizable  over  P  are  independently 
distributed  in  (— u,  u]  and  [n  —  u,  n-j-  u).  These  numbers  have  about  the 
same  frequency  in  (n  —  u,  n  -f-  u]  and  [0,  n)  for  0  <  <  u  <  <  n. 

In  particular  (A2)  implies 

#B  >  iP{n^2r,n^2r)-rp(n,n^2r)/n  >  nd/2r(\n  n)~2r~d. 

Observe  that 

ip(n,  nl/T)  ;==  #{u;  £  {1,  n]:  all  prime  factors  of  w  <  v} 

>f(n,/r')  +  r)>n(bn)-r  +  2. 

Let  7Xr)  be  the  time  of  our  algorithm.  Then  (Al),  (A2)  imply 

Fact  7.  T(n)  =  0(nd/2rlnn  +  n3/2f)  provided  nd^2r(lnn)~2r~d  >  2 nl/2r. 

Proof.  According  to  (Al)  and  (A2),  the  relation  nd/2,(\nn)~2r~d  >  2 nl/2r 
implies  #5  >  2(^(v)-j-l)  and  therefore  implies  the  solvability  of  the  linear  system 
(2). 

0(nrf/2rlnn)  bounds  the  steps  to  generate  L,  L,  and  B,  if  we  compute  L 
(and  similarly  L)  as  follows.  The  prime  factors  <  n1/3r  of  w  are  collected  in  Lw. 


for  all  v)  with  |n  —  w\  <  nd/2r  do  Lw  :=  0 
for  all  p  6  P  and  all  v  with  \v\  <  nd^2r/p  do 
[insert  p  into  f/n-j~t'p— nmodp] 
for  every  w  and  every  p;  6  Lw  do 
[«.•(«»)  :=  max{i/:  p*  |  tn}] 

L  :=  {(w,  (a,(ty):  t  <  *(»))  1  tt»  =  II,^w(.)Pj,(w>> 

0(n3/2f)  bounds  the  number  of  steps  to  solve  the  linear  system  (2).  Q 

In  order  to  minimize  our  time  bound  we  choose  d,  r  such  that  nd!2r  « 
2(ln  n)2,+dn1/2r.  This  yields 


2r  fa  Vd—  1  \  — —  provided  d  <<  r. 

V  In  Inn 

This  yields  for  d  =  3: 


T(n)  =  0(exp  \/4.5  In  n  In  In  n ). 

This  means  that  our  algorithm  is  asymptotically  superior  to  Dixon’s  algorithm, 
but  inferior  to  the  Brillhart-Morrison  method.  So  far  we  have  proved: 

Theorem  2.  [Assume  (Al),  (A2).J  The  above  algorithm  has  time  bound 

0(exp\/4.51nr.lnlnn). 


One  interesting  feature  of  the  above  algorithm  is  that  the  main  work  in  stage 
1,  namely  the  construction  of  the  lists  L,  L  is  almost  independent  from  n.  These 
lists  can  be  used  uniformly  for  the  factorization  of  all  numbers  in  (n  —  u,  n  -f-  u], 
u  =  In  particular,  if  someone  has  factored  n  he  already  has  collected 

the  data  to  easily  factor  each  number  near  to  n.  Considering  the  problem  of 
factorizing  many  numbers  in  [n  —  u,  n+  u]  we  will  assume  that  the  lists  L,  L  are 
built  up  once  for  ever  and  that  they  are  sorted  with  respect  to  the  first  component 
of  the  elements  (w,o)  and  (n-hw,£).  Under  this  assumption  we  will  now  bound 
the  remaining  number  of  steps. 
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Given  L  and  L  we  can  form  a  sufficiently  large  subset  B  of  B  as  follows: 
B  :=  0 

while  #£  <  2(7r(t»)  -f  1)  do 

begin  choose  (n  +  w,  6)  6  L  at  random 
eliminate  (n  +  w,b)  from  L 
if  («;,  s)  E  L  for  some  a  then 
(insert  (a,  b)  into  B] 

end 

It  follows  from  (A2)  that  this  will  take 

0(^(v)(lnn)<i}  =  0(n1/2f(lnn)‘i) 
steps.  This  yields  a  total  time  bound  as 

T(«)  =  0(n1/2r(ln»)*  +  n3/2r) 

for  all  r,  d  with  nd/2r(\nn)~2r~~d  >  2 n1/2' .  We  choose  r,  d  such  that 

nrf/2,(lnn)-2f-<<  «  2 nl/2f 


which  yields 


2r  «  'Jd—l  \  — —  provided  d  < < 

V  lnlnn 


Then  minimizing  the  time  bound  with  respect  to  d  yields 


=2W'JejlY 

Vlnlnn/ 


and  the  corresponding  time  bound  is 


T(n)  =  0(exp(2(lnn)l/3(lnlnn)2/3). 


Thus  we  have  proved: 


Theorem  3.  (Assume  (Al),  (A2).J  Given  L,  L,  the  time  bound  of  the  algorithm 
is 

T(n)  =  0(exp(2(lnn)1</3(inlnn)2/3)). 
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This  theorem  can  be  interpreted  as  follows.  Suppose  we  like  to  factor  all 
numbers  in  [n-u(n  +  u],u  =  nd/2r  and  let  the  cost  to  preprocess  the  lists  L,  L 
be  unformly  distributed  to  the  numbers  in  (n  —  u,  n  -}•  u].  Then  the  factorization 
of  every  specific  number  in  [n— u,  n-f-u]  accounts  for  0(exp[2(ln  n)1/3(ln  In  n)2/3)) 
steps. 

We  observe  that  the  improvement  by  preprocessing  the  lists  L  and  L  can 
even  be  strengthened,  if  we  also  preprocess  for  various  A’s  the  lists  of  all  numbers 
in  [Jfcn  —  u,  Jfcn  -f  u]  which  are  completely  factorizable  over  P. 
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4.  Improvements  on  a  Method  of  Shanks. 

Shanks  (1971)  proposed  a  factoring  method  which  starts  by  computing  the 
group  of  equivalence  classes  of  primitive  quadratic  forms  with  discriminant  — n 
and  in  particular  he  computes  the  order  h(— n)  of  this  group.  Then  he  factors  n 
by  constructing  a  non-trivial  ambiguous  class.  Under  the  implicit  assumption  that 
the  entire  group  of  classes  is  generated  by  small  “prime"  forms,  and  by  neglecting 
logn  factors,  Shanks  proves  a  time  bound  of  about  Ofa1/4).  Monier  (1980)  claims 
that  this  time  bound  can  be  improved  to  ©(n1/5)  under  the  assumption  of  the 
generalized  Riemann  hypothesis.  He  claims  that  under  this  hypothesis  the  well 
known  convergence 


=  h(—n) 


has  an  error  term  0 (nl/2m“ */2)  which  would  speed  up  the  evaluation  of  h( — n). 

We  propose  a  way  to  construct  ambiguous  classes  without  evaluating  h( — n) 
at  all.  We  exploit  the  fact  that  ambiguous  forms  can  be  constructed  mainly 
in  the  same  way  as  we  generate  solutions  of  x2  —  j/2modn,  by  the  method  of 
combining  congruences.  Under  reasonable  assumptions  this  yields  an  asymptotical 
time  bound  0(exp\/31nnlnlnn). 

We  summarize  some  basic  facts  on  binary  quadratic  forms.  We  find  it  most 
convenient  to  follow  the  original  presentation  of  Gauss  (1801,  1889)  which  slightly 
differs  from  that  of  Shanks  (1971).  The  form  az2-\-2bzy-{-cy2  with  a,  6,  c  £  Z  will 
be  described  by  the  triple  ( a,b,c ).  Two  forms  (a,  6,  c)  and  (a,  6,  l)  are  equivalent 
if  there  exist  linear  transformations  with  integer  coefficients  and  determinant 

1  transforming  the  one  form  into  the  other;  i.e.,  T~ ^7*  =  ^  for 

some  integer  matrices  T,  T~ 1  with  dctT  =  1.  Equivalent  forms  have  the  same 
determinant  D  :=  b2— i :.  A  form  (a,  6,  c)  is  (properly)  primitive  if  gcd(a,  26,  c)  = 
1.  According  to  Gauss,  the  non-primitive  forms  can  all  be  derived  from  primitive 
ones.  Therefore  it  is  most  important  to  understand  the  structure  of  the  primitive 
forms. 

Henceforth  we  will  restrict  all  considerations  to  forms  with  negative  deter¬ 
minants  D  =  62  —  ac  <  0.  In  this  case  the  equivalence  classes  can  be  charac¬ 
terized  by  reduced  forms.  A  form  (a,  6,  c)  is  reduced  if  2|6|  <  |a|  <  |c|.  There 
is  a  jfed-like  algorithm  which,  given  (a,  6,  c)  computes  an  equivalent  reduced  form 
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within  0(ln|a6c|)  arithmetical  steps: 


while  (a,  b,  c)  is  not  reduced  do 

begin  5  :=  —b mode  with  |6|  <  c/2 
(a,  i,  e)  :=  (c,t ,(?  -  D)/c) 

end 

Theorem  i.  [Gauss,  Artikel  172.J  In  every  equivalence  class  H  with  D  <  0  there 
is  either  exactly  one  reduced  form  (a,  b,  c )  or  exactly  two  reduced  forms  (a,  ±;b,  c). 
In  the  latter  case,  H  is  called  ambiguous. 

A  form  with  D  <  0  either  satisfies  a,  c  >  0  or  a,  c  <  0.  It  is  called  positive  in 
the  first  and  negative  in  the  second  case.  Positive  (negative,  resp.)  forms  ax2  + 
2 bxy  cy 2  only  take  positive  (negative,  resp.)  values  for  real  x,  y  (which  follows 
from  ac  >  b2).  Since  this  property  is  preserved  under  the  equivalence  relation,  a 
class  must  be  either  positive,  containing  only  positive  forms,  or  it  must  be  negative 
and  contains  only  negative  forms.  Moreover  there  is  a  one-one  correspondence 
between  the  positive  and  the  negative  forms  as  (a,  6, c)  ~  (— a, 6,  — c).  Therefore 
we  can  w.l.o.g.  restrict  our  considerations  to  positive  forms  and  these  generate 
exactly  half  of  the  equivalence  classes.  The  number  of  equivalence  classes  with 
determinant  D  is  finite  since  a  reduced,  positive  form  (a,  6,  c)  always  satisfies 
2|6|  <  a  <  y/4\D\/3. 

Gauss  (1801)  introduced  the  composition  of  (binary)  quadratic  forms  and 
proved  that  the  equivalence  classes  with  fixed  determinant  D  form  an  abelian 
group,  say  QF(D),  under  composition.  Given  two  classes  H\,  Hz  represented 
by  their  reduced  forms,  the  reduced  form  of  H\  •  Hz  can  be  computed  within 
0(ln|Z)|)  arithmetical  steps  over  numbers  <  |Z?|.  The  forms  which  are  primitive 
and  positive  generate  a  subgroup  of  QF(D )  which  we  call  QFP(D).  The  unit 
element  I  of  the  group  is  represented  by  (1,0,  —D). 

The  following  assertions  are  equivalent:  (1)  H  is  ambiguous,  (2)  H-H  —  I, 

(3)  every  form  (a,6,c)  in  H  is  equivalent  to  ( a,—b,c ),  (4)  — 

^  for  some  integer  matrices  T,  T~l  with  detT1  =  —1. 

The  reduced  form  of  an  ambiguous  class  is  of  either  of  the  following  three 
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air 


t 


types: 


5  =  0  or  a  —  2b  or  a  =  c. 


We  call  these  forms  ambiguous,  they  always  represent  ambiguous  classes.  These 
three  types  of  ambiguous  forms  yield  the  folowing  factorizations  of  the  deter¬ 
minant: 


—D  =  ac  ,  — D  =  t(2c  —  6)  ,  —D  =  (a  —  b)(a  +  b). 

In  this  way  the  problem  of  factoring  n  reduces  to  the  construction  of  am¬ 
biguous  forms  with  determinant  —n.  It  is  important  that  Gauss  has  established  a 
strong  correspondence  between  the  factorizations  of  n  and  the  ambiguous  classes 
in  QFP{-n). 

We  only  report  the  case  n  odd,  since  we  like  to  factor  only  odd  nubers. 

A  pair  (nlf  n2)  £  )J2  is  an  admissible  factor  pair  for  n  if  n  =  nj  •  n2,  Mi  < 
«2  and  gcd(ni,n2)  =  1.  Suppose  n  has  (exactly)  l  distinct  prime  factors,  then 
there  are  (exactly)  2L~l  admissible  factor  pairs  for  n. 

Theorem  5.  [Gauss,  Artikel  257,  258.]  Suppose  n  £  M  is  odd  and  has  l  >  1 
distinct  prime  factors.  Then  there  are  2t'~ 1  or  2l  ambiguous  classes  in  QFP(—n) 
according  to  whether  n  =  3  mod  4  or  n  =  1  mod  4.  Each  of  the  2l~l  admissible 
factor  pairs  of  n  is  obtained  by  the  reduced  form  of  exactly  one  in  case  n  = 
3  mod  4  (two  in  case  n  =  I  mod4j  of  these  ambiguous  classes. 

Example.  We  list  n:  all  ambiguous  forms  with  determinant  —  n  and  b  >  0  that 
are  primitive,  reduced,  and  positive;  the  corresponding  list  of  admissible  factor 
pairs. 

n  =  3:  (1,0,3);  (1,3) 

n  =  5:  (1,0,5),  (2, 1,3);  (1, 5),  (1,5) 
n  =  15:  (1,0,15),  (3,0,5);  (1,15),  (3,5) 

n  =  21:  (1,0,21),  (3,0,7),  (2,1,11),  (5,2,5);  (1, 21),  (3,7),  (1,21),  (3,7) 
n  =  105:  (1,0, 105),  (3, 0,35),  (5, 0,21),  (7, 0,15),  (2, 1,53),  (6,3, 19),  (10,  5, 13), 
(11,4,11);  (1,105),  (3,35),  (5,21),  (7,15),  (1,105),  (3,35),  (5,21), 
(7,15). 

The  distinction  between  the  cases  n  =  1  mod  4  and  n  =  3  mod  4  is  explained 
as  follows.  The  ambiguous  and  reduced  form  (2,  l,(n-f- 1)/2)  is  primitive  in  case 
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n  =  1  mod  4  whereas  it  is  imprimitive  in  case  n  =  3  mod  4,  since  in  the  latter 
case  gcd(2, 2,  (n  -f  l)/2)  =  2.  Since  the  product  of  two  ambiguous  classes  is  again 
ambiguous,  there  must  be  twice  as  many  ambiguous  classes  in  case  n  =  1  mod  4 
as  there  are  in  case  n  =  3  mod  4. 

The  remaining  point  to  be  discussed  for  the  factorization  of  n  is  how  to 
generate  ambiguous  classes  in  QFP( — n).  This  will  be  done  by  exploiting  the 
group  structure  of  QFP( — n).  Let  H,R  6  QFP{ — n)  be  represented  by  (a,  6,  c) 
and  (a.M),  i-e.,  H  =  [(a,6,c)),  R  =  ((2,6,2))-  Then  by  definition  a  repre¬ 
sentative  ( A,B,C )  for  H  -R  can  be  found  as  follows: 

fi  :=  gcd(a,  2,  6  -f-  6) 

Compute  a,P,  7  6  2  such  that 
aa  +  p  a  +  7(6  +  6)  =  /i. 

A  :=  aa/n2 

B  :=  [aa6  +  Pab  -f-  7(66  —  n)]//mod  A 
C  :=  (n  +  B2)/A 

In  the  special  case  that  gcd(a,  5)  =  1  one  obtains  in  this  way  (observe  that 
we  can  choose  7  =  0  and  a,  P  such  that  aa  -f-  /?S  =  1): 

A  :=  aa 

Choose  B  such  that 
B  —  b  mod  a  and  B  =b  mod  a 
C  :=  (n  +  B2)JA. 

(A,  B,  C)  will  be  primitive  but  not  necessarily  reduced.  \(A,  B,  C)\  does  not 
depend  on  the  distinct  possible  choices  for  ct,  P,  7<  6?,  C.  Since  a,  P ,  7 
can  be  computed  via  Euclid’s  gcd-algorithm,  it  is  clear  that  this  multiplication 
scheme  requires  only  0(ln  n)  arithmetical  steps  over  numbers  <  O(n)  provided 
(a,  6,  c)  and  (2,6, c)  are  reduced.  It  can  easily  be  seen  that 

l(«A  «))((«» “M)l  =  l- 


In  this  case  n  =  a,  A  =  1  and  the  choice  a  =  l,/?  =  7  =  °  yields  B  =  ab  and 
therefore  A  |  B.  Then  A  =  1  implies  ((A,  B,  C))  =  I. 

The  special  case  ged (a,  2)  =  1  of  this  multiplication  scheme  immediately 
implies  the  following. 
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Fact  8.  Let  ((a,  6,  c)]  £  QFP(—n)  and  Jet  a  =  n,p“‘  the  prime  factorization 
of  a,  then  ((a,6,c)]  =  II, ((p“‘.  *•»«•)]  with  bi  ■=  bmodpf*  and  a  :=  (b?+n)/p?1. 

The  possibly  occuring  factors  ((pf'.i,*,  e,)]  in  Fact  8  can  be  characterized  as 
follows. 

Lemma  3.  Let  p  be  prime,  p  ^  2,  gcd(p,  n)  =  1  and  a  >  1 .  There  exists 
[(p°,6,  c)]  €  QFP(—n)  with  integers  b,  c  iff  (=j&)  =  1.  If  (=■*)  =  1  there 
are  exactly  two  of  these  classes,  namely  ^“,±6,  (n  +  fc^/p6")]  for  b  with  b2  = 
— nmodp01. 

Proof.  Suppose  (p01, 6t  c)  is  a  positive  form  with  determinant  — n.  Then 
— n  =  b2  — pac  which  means  that  — n  is  a  quadratic  residue  mod  p“.  Hence 
(pP)  =  (=*)  =  1.  There  are  exactly  two  square  roots  ±6  of  —nmodp01.  The 
classes  [(pa,  (n-f-  b2)/pa)\  are  distinct  and  primitive.  In  fact  these  classes  are 

inverse  and  non-ambiguous,  since  gcd(p,  n)  =  1,  p  ^  2.  Q 

We  denote  one  of  the  classes  [(p01,  ±6,  (n  +  b2)/pa)]  occuring  in  Lemma  3  as 
7P*  n.  Then  the  other  class  must  be  {Ip*,n)~l-  It  is  clear  from  the  multiplication 
scheme  that 

{(fp.n)01,  [Ip,n)  a)  —  {Ip*,n,  {Ipa,n)  *}• 

This  implies  that  Fact  8  can  be  rewritten  as  follows. 

Lemma  4.  Let  [(a,  6,  c)]  6  QFP{— n),  a  odd  and  let  a  =  H  P?‘  be  the  prime 
factorization  of  a.  Then 

k«,  »,  c»  =  ntwr"  «.  =  ±i. 

I 

In  particular,  factoring  [(a,  b,  c)]  6  QFP(-n)  as  in  Lemma  4  can  be  done 
roughly  in  the  time  which  is  necessary  to  factor  a.  Since  we  know 

(WP“  =  to*f  *. 

with  b{  =  6modp?‘,  c,-  =  ( 6 2  +  n)/p®*,  we  can  easily  check  whether  c,-  =  1  or 
€,*  =  —1.  Also,  in  the  case  that  a  is  even,  c  must  be  odd  provided  ( a,b,c )  is 


primitive.  Hence,  if  o  is  even  we  can  apply  Lemma  4  to  the  form  (e,  —b,  a)  which 
is  equivalent  to  (a,b,c). 

By  means  of  Lemma  4  we  can  generate  ambiguous  forms  with  determinant 
— n  mainly  in  the  same  way  as  congruences  x2  =  y2modn  are  produced  by 
Dixon's  factoring  algorithm. 


Construction  of  ambiguous  classes  in  QFP(—n). 

stage  1  Construct  a  factor  base 

P  :=  {p  |  2  <  p  <  t>,  p  prime,  (=*)  =  1} 

if  3p  E  P:  p  |  n  then  print  p  stop 

for  all  p  E  P  compute  Ip  :=  (p,  6,  ( b 2  +  n)/p ) 

comment  [We  discuss  the  optimal  choice  of  v  below.  Compute  Ip  by  solving 
b2  =  — -nmodp  using  the  probabilistic  algorithm  of  Berlekamp, 
Rabin,  see  Rabin  (1979).] 


stage  2  Choose  a  random  H  €  QFP(—n)  which  is  generated  by  the  Ip  with 
pE  P  (i.e.,  compute  H  =  Il^ep  with  a  random  (a,- 1  p{  E  P)  E 
such  that  £tlna,-  <  (Inn)2] 

Compute  the  reduced  form  (a,  b,c)  of  H  •  H. 

Try  to  factor  a  over  P  and  [(c,6,c)]  over  {Ip  |  p  E  P}. 
if  H2  =  JI {I pi  w*th  a,-  E  Z  then 

[store  fl  =  (a,-  ]  p,  6  P)  and  set  :=  H ] 
while  <  #P  vectors  a  have  been  found  goto  stage  2 
Solve  £a  /ao  =  0  mod 2  with  /a  E  {0, 1} 

Then  II/A=i  ^  >san  ambiguous  class. 


comment 


Observe  that  the  construction  implies  II/A=i  —  n P(ep  I 


pi 


Even  if  n  =  1  mod  4  we  do  not  include  p  =  2  into  the  factor  base  P,  since 
the  ambiguous  class  I2  —  ((2, 1,  (n  +  l)/2)]  corresponds  to  the  trivial  factor  pair 
(!*»)• 
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Example,  n  =  1037 

We  choose  the  factor  base  P  =  {3, 13},  we  have  =  (ti1)  = 

— 1.  The  corresponding  classes  are 

h  —  [(3, 1,346)]  ,  /i3  =  ((13,4,81)]. 


One  obtains 


Hence  /13  -/3  1  is  ambiguous.  The  reduced  form  in  this  class  is  (34, 17,39)  which 
yields  the  factorization 

1037  =  17(78-  17)  =  17-61. 

Observe  that  the  factor  base  in  this  example  is  smaller  than  in  the  application  of 
Miller’s  method  in  Section  3.  Dixon’s  algorithm  would  require  a  larger  factor  base 
too.  Indeed  the  factor  base  is  so  small  since  the  primes  p  =  5,7, 11  are  excluded 
because  (==*)  —  — 1. 

In  our  analysis  of  the  algorithm  we  will  use  the  following  heuristic  assump¬ 
tions. 

(A3)  #{p  <  v:  p  prime,  =  l}  >  7^7  with  c  >  0  fixed. 

(A4)  every  admissible  factor  pair  of  n  corresponds  to  some  ambiguous  class 
which  is  generated  by  the  Iv,  p  <  v. 

Assumption  (A3)  certainly  fails  for  a  few  n  but  it  must  hold  for  most  n  since 
we  have: 

£_  *{p  <  *  -  p  (?)  =  >}  «  2ib- 

This  follows  from  ir(v)  «  v/lnv  and  from  the  fact  that  (®)  =  1  for  exactly  half 
of  the  a  £  Zp.  We  argue  that  this  supports  (A3)  since  we  can  as  well  apply  our 
algorithm  to  factor  any  number  n-k,  k  odd,  k  <  <  n.  Then  factors  of  n  will  be 
found  with  the  same  probability  as  those  of  k. 

The  assumption  (A4)  is  still  somewhat  weaker  than  the  assumption  used  by 
Shanks  (1971)  that  the  whole  group  GFP(—n)  is  generated  by  the  classes  lp  with 
small  p. 
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Under  the  assumptions  (A3),  (A4)  the  analysis  of  the  algorithm  becomes 
virtually  very  similar  to  the  analysis  of  Dixon’s  algorithm.  The  main  advantage 
over  Dixon’s  algorithm  is  that  we  have  to  factor  numbers  a  =  O (y/n),  instead 
of  numbers  w  =  O(n),  over  the  base  of  small  primes.  Therefore  we  can  argue  as 
in  the  case,  that  quadratic  residues  mod  n,  w  =  0(\/n)  are  constructed  by  the 
continuous  fraction  method,  see  the  send  of  Section  2.  We  choose 


v  ==  n^2f  ,  r  —  2 


1 

/31nn 

4  \ 

1  In  Inn 

and  obtain  n  as  a  final  result. 

Theorem  4.  / Assume  (A3),  (A4).)  Suppose  we  factor  a  composite  n  via  the 
construction  of  ambiguous  forms  with  determinant  — n  as  above,  then  for  each  n  a 
proper  factor  of  n  will  be  found  with  probability  1/2  within  0(expv/31nnlnlnn) 
steps. 

The  above  factoring  method  can  be  interpreted  as  the  continuous  fraction 
method  in  case  of  negative  determinants.  Conversely,  in  case  of  positive  deter¬ 
minants  D  =  b2  — a c  >  0,  there  is  a  different  concept  of  reduced  forms  and  there 
are  many  equivalent  reduced  forms.  According  to  Gauss,  Artikel  183-187,  the 
equivalent  reduced  forms  can  be  developed  into  an  even  and  symmetric  period. 
The  recursion  for  developing  this  period  is  the  same  as  that  for  evaluating  the 
period  of  the  continuous  fraction  of  VD.  Shanks  exploited  this  coincidence  and 
proposed  an  algorithm  to  factor  n  by  constructing  an  ambiguous  form  with  posi¬ 
tive  determinant  n.  Shanks  has  a  way  to  make  giant  steps  within  the  period  of 
equivalent  reduced  forms  (this  is  used  in  order  to  decide  whether  two  forms  are 
equivalent).  This  second  algorithm  of  Shanks  runs  in  about  OJn1/4)  steps,  see 
Monier  (1980)  for  a  more  detailed  exposition  of  this  method. 
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